Decoding device, decoding method, program, encoding device, and encoding method

ABSTRACT

An object is to reduce the amount of data to be transmitted while ensuring tactile reproducibility. A decoding device according to the present technology includes: a first decoding unit that decodes first encoded data obtained by encoding a first signal section with a first bit rate, the first signal section being a part of a touch signal section which is a signal section indicating a touch state with an object in a tactile signal, the first signal section being a signal section including a boundary between the touch state and a non-touch state with the object; and a second decoding unit that decodes second encoded data obtained by encoding a second signal section with a bit rate lower than the first bit rate, the second signal section being a signal section except for the first signal section in the touch signal section.

TECHNICAL FIELD

The present technology relates to a decoding device that decodes anencoded tactile signal, a decoding method thereof, a program of thedecoding device, an encoding device that encodes a tactile signal, anencoding method thereof, and a program of the encoding device.

BACKGROUND ART

In recent years, there has been developed a technology in which a devicemounted on a user vibrates to give a tactile stimulus to the user. Here,the tactile stimulus refers to a physical phenomenon that causes theuser to perceive a tactile sensation due to a vibration phenomenon orthe like. In addition, generating a tactile stimulus is referred to as atactile presentation.

Technologies for tactile presentation are used in devices in variousfields. For example, in a terminal device with a touch panel such as asmartphone, the touch panel vibrates in response to a touch operationfrom the user and gives a tactile stimulus to the user's finger, so thatthe user can feel a touch on a button or the like displayed on the touchpanel. Further, for example, in a music listening device such asheadphones,

a tactile stimulus is given in time with music playback, so that thedeep bass in the music being played back can be boosted. Further, forexample, in a device that provides a computer game or VR (VirtualReality), a controller or the like is vibrated according to an operationof the controller or a scene of the content to give a tactile stimulusto the user, so that the immersive feeling of the user in the contentcan be enhanced.

In addition, there has been developed a technology for giving a tactilestimulus to a user based on a tactile signal received from an externaldevice. For example, PTL 1 referred to below discloses a technology forgiving a tactile stimulus to a user with the frequency and amplitude ofvibration being changed based on a received signal.

CITATION LIST Patent Literature

[PTL 1]

-   JP 2016-202486 A

SUMMARY Technical Problem

Here, it is considered that when performing a tactile presentation for aplurality of parts of the user's body, it is necessary to prepare atactile signal for each of the channels corresponding to the respectiveparts of the user. In this case, it is assumed that the total amount oftactile signal data required increases as the number of parts to whichtactile stimuli are given increases. In addition, even when thefrequency range of the vibration included in the tactile signal iswidened, the total amount of tactile signal data required increases.Therefore, the transmission and reception of tactile signals may bedelayed due to the congestion state of a network used for thetransmission and reception of the tactile signals.

The present technology has been made in view of the above circumstances,and aims to achieve both a reduced amount of data to be transmitted anda guarantee of reproduction of tactile sensation.

Solution to Problem

A decoding device according to the present technology includes: a firstdecoding unit that decodes first encoded data obtained by encoding afirst signal section with a first bit rate, the first signal sectionbeing a part of a touch signal section which is a signal sectionindicating a touch state with an object in a tactile signal, the firstsignal section being a signal section including a boundary between thetouch state and a non-touch state with the object; and a second decodingunit that decodes second encoded data obtained by encoding a secondsignal section with a bit rate lower than the first bit rate, the secondsignal section being a signal section except for the first signalsection in the touch signal section. As a result, it is possible toappropriately decode encoded tactile data in which only the significantsection in the touch signal section is encoded with the high bit rateand the other section is encoded with the low bit rate, so that both areduced amount of data to be transmitted and a guarantee of reproductionof tactile sensation can be achieved.

The decoding device according to the present technology described abovemay include a combining unit that combines first waveform data obtainedby decoding the first encoded data and second waveform data obtained bydecoding the second encoded data.

As a result, the pieces of waveform data are integrated into one pieceof waveform data.

In the decoding device according to the present technology describedabove, the combining unit may perform cross-fade processing on acombined portion between the first waveform data and the second waveformdata.

As a result, the first waveform data and the second waveform data aresmoothly combined.

A decoding method according to the present technology includes: decodingfirst encoded data obtained by encoding a first signal section with afirst bit rate, the first signal section being a part of a touch signalsection which is a signal section indicating a touch state with anobject in a tactile signal, the first signal section being a signalsection including a boundary between the touch state and a non-touchstate with the object; and decoding second encoded data obtained byencoding a second signal section with a bit rate lower than the firstbit rate, the second signal section being a signal section except forthe first signal section in the touch signal section.

With such a decoding method, the same operation as that of theabove-described decoding device according to the present technology canalso be obtained.

A first program according to the present technology is a program causingan information processing device to execute: a function of decodingfirst encoded data obtained by encoding a first signal section with afirst bit rate, the first signal section being a part of a touch signalsection which is a signal section indicating a touch state with anobject in a tactile signal, the first signal section being a signalsection including a boundary between the touch state and a non-touchstate with the object; and a function of decoding second encoded dataobtained by encoding a second signal section with a bit rate lower thanthe first bit rate, the second signal section being a signal sectionexcept for the first signal section in the touch signal section.

Such a first program according to the present technology realizes theabove-described decoding device according to the present technology.

An encoding device according to the present technology includes: adetermination unit that determines a first signal section and a secondsignal section, the first signal section being a part of a touch signalsection which is a signal section indicating a touch state with anobject in a tactile signal, the first signal section being a signalsection including a boundary between the touch state and a non-touchstate with the object, the second signal section being a signal sectionexcept for the first signal section in the touch signal section; a firstencoding unit that encodes the first signal section with a first bitrate; and a second encoding unit that encodes the second signal sectionwith a bit rate lower than the first bit rate. As a result, only thesignificant section in the touch signal section is encoded with the highbit rate, and the other section is encoded with the low bit rate.

In the encoding device according to the present technology describedabove, the determination unit may determine the first signal section andthe second signal section based on section information on the firstsignal section and the second signal section added to the tactilesignal.

As a result, waveform analysis processing is not necessary indetermining the first signal section and the second signal section.

In the encoding device according to the present technology describedabove, the determination unit may determine the first signal section andthe second signal section based on a result of performing a waveformanalysis on the tactile signal. As a result, information fordifferentiating between the first signal section and the second signalsection is not required to be added to the touch signal in individuallyencoding the first signal section and the second signal section.

In the encoding device according to the present technology describedabove, the determination unit may determine the first signal section andthe second signal section based on an amplitude change rate of thetactile signal.

As a result, it is possible to appropriately determine the first signalsection with a sharp change in the waveform of the tactile signal.

In the encoding device according to the present technology describedabove, the second encoding unit may encode the second signal section byan encoding method using a longer conversion length than in encoding forthe first signal section.

As a result, the amount of second encoded data in the second signalsection is reduced.

In the encoding device according to the present technology describedabove, the second encoding unit may perform parametric encoding on thesecond signal section.

As a result, the amount of second encoded data in the second signalsection is reduced.

The encoding device according to the present technology described abovemay include a first buffer memory used for determining the first signalsection and the second signal section, and a second buffer memory usedfor encoding the second signal section according to a result ofdetermining.

As a result, it is possible for a second buffering processing unit tobuffer a tactile signal that fails to be buffered by a first bufferingprocessing unit for the tactile signal in the second signal section forwhich the conversion length is longer than in the encoding for the firstsignal section.

An encoding method according to the present technology includes:determining a first signal section and a second signal section, thefirst signal section being a part of a touch signal section which is asignal section indicating a touch state with an object in a tactilesignal, the first signal section being a signal section including aboundary between the touch state and a non-touch state with the object,the second signal section being a signal section except for the firstsignal section in the touch signal section; encoding the first signalsection with a first bit rate; and encoding the second signal sectionwith a bit rate lower than the first bit rate.

With such an encoding method, the same operation as that of theabove-described encoding device according to the present technology canalso be obtained.

A second program according to the present technology is a programcausing an information processing device to execute: a function ofdetermining a first signal section and a second signal section, thefirst signal section being a part of a touch signal section which is asignal section indicating a touch state with an object in a tactilesignal, the first signal section being a signal section including aboundary between the touch state and a non-touch state with the object,the second signal section being a signal section except for the firstsignal section in the touch signal section; a function of encoding thefirst signal section with a first bit rate; and a function of encodingthe second signal section with a bit rate lower than the first bit rate.

Such a second program according to the present technology realizes theabove-described encoding device according to the present technology.

BRIEF DESCRIPTION OF DRAWINGS

FIG. 1 is a diagram illustrating a configuration example of a tactilereproduction system according to an embodiment of the presenttechnology.

FIG. 2 is a diagram illustrating a configuration example of an encodingdevice according to an embodiment.

FIG. 3 is a diagram illustrating a configuration example of areproduction device according to an embodiment.

FIG. 4 is a diagram illustrating a configuration example of a decodingdevice according to an embodiment.

FIG. 5 is a diagram illustrating a usage example of the tactilereproduction system according to an embodiment of the presenttechnology.

FIG. 6 is a diagram illustrating a vibration detection threshold curvethat serves as a guideline for human tactile sensitivity according to anembodiment.

FIG. 7 is a diagram illustrating a vibration waveform of a tactilesignal according to an embodiment.

FIG. 8 is a diagram illustrating a configuration example of an encodingunit according to an embodiment.

FIG. 9 is a diagram illustrating a configuration example of a dataformat of encoded tactile data according to an embodiment.

FIG. 10 is a flowchart illustrating processing to realize an encodingmethod according to an embodiment.

FIG. 11 is a diagram illustrating a configuration example of a dataformat of encoded tactile data according to an embodiment.

FIG. 12 is a flowchart illustrating processing to realize an encodingmethod according to an embodiment.

FIG. 13 is an illustrative diagram of cross-fade processing performed bya combining unit according to an embodiment.

FIG. 14 is a flowchart illustrating processing to realize a decodingmethod according to an embodiment.

FIG. 15 is a detailed diagram of cross-fade processing performed by acombining unit according to an embodiment.

FIG. 16 is a detailed diagram of cross-fade processing performed by thecombining unit according to the embodiment.

FIG. 17 is a diagram illustrating a configuration example of a firstapplication example of an embodiment.

FIG. 18 is a diagram illustrating a configuration example of a secondapplication example of an embodiment.

FIG. 19 is a diagram illustrating a configuration example of a thirdapplication example of an embodiment.

DESCRIPTION OF EMBODIMENTS

Embodiments of the present technology will be described with referenceto the above-mentioned drawings The description of the drawings for theconfiguration, if previously described, may be referred to with the samereference numerals and the description thereof may be omitted. Inaddition, the drawings are schematic, and the main parts necessary fordescribing the present technology and the related configurations areextracted for illustration. The relationship between the thickness,plane dimensions, ratios, and the like of the structures illustrated inthe drawings are merely examples, and various changes can be madeaccording to, for example, their designs, as long as they do not departfrom the technical idea of the present technology.

Hereinafter, an embodiment will be described in the following order.

<1. Overview of tactile reproduction system>

<2. Configuration of encoding device>

<3. Configuration of reproduction device>

<4. Configuration of decoding device>

<5. Usage examples of tactile reproduction system>

<6. Tactile reproduction method as embodiment>

[6-1. Issues related to tactile signal transmission]

[6-2. Encoding method]

[6-3. Decoding method]

<7. First application example>

<8. Second application example>

<9. Third application example>

<10. Conclusion>

<11. Present technology>

The terms as used herein are now defined as follows.

Tactile stimulus: A physical phenomenon that causes a person to perceivea tactile sensation, such as a vibration phenomenon.

Tactile presentation: Generating a tactile stimulus.

Tactile signal: A signal that represents a pattern of tactile stimulus,such as a signal that represents a vibration waveform.

Recipient: A person who receives a tactile presentation.

Tactile characteristics: Human tactile characteristics. It depends on apart (hand, face, foot, etc.).

Tactile sensitivity: A sensitivity of how strong the tactile stimulus issubjectively perceived. It depends on a receptor or part of the humanbody. A high tactile sensitivity means that it is easy to perceivetactile signals.

Encoded tactile data: Data in which a tactile signal is encoded. Itincludes a stream and a frame, which are subordinate concepts. Theencoded tactile data includes first encoded data and second encodeddata, which will be described later.

1. Overview of Tactile Reproduction System

FIG. 1 illustrates a configuration example of a tactile reproductionsystem 1 including an encoding device 2 and a decoding device 3, as anembodiment according to the present technology.

First, in the present embodiment, environment for realizing tactilereproduction is classified into: a collection environment in which atactile signal obtained by sensing tactile information (tactilestimulus) as a target is encoded, and the encoded tactile data Dcobtained by encoding is collected; and a reproduction environment inwhich the tactile information is reproduced based on a tactile signalobtained by decoding encoded tactile data Dc.

As illustrated, the tactile reproduction system 1 includes in thecollection environment a plurality of tactile sensors 5 and an encodingdevice 2 to which the tactile sensors 5 are connected, and includes inthe reproduction environment a reproduction device 4 configured toacquire encoded tactile data Dc, a decoding device 3 configured toenable wireless communication with the reproduction device 4, and aplurality of tactile presentation devices 6 connected to the decodingdevice 3.

Each tactile sensor 5 is a sensor that senses a tactile stimulus, and inthe present example, a vibration sensor such as a piezo pickup or anacceleration sensor is used. The tactile sensor outputs vibration ormotion as a voltage change when brought into touch with an object to besensed, that is, the human body in the present example.

In the present example, the tactile sensors 5 are in wired connection tothe encoding device 2, and the tactile sensors 5 are mounted ondifferent parts of the human body as the object and senses a tactilestimulus generated at each part. The parts on which the tactile sensors5 are mounted are not limited to those of the human body, and a tactilesensor 5 may be mounted on a tool or the like used in the tactilereproduction system 1. In this case, the tactile sensor 5 senses atactile stimulus generated at the corresponding part on which the toolas an object is mounted.

The encoding device 2 is configured to include, for example, a computerdevice such as a central processing unit (CPU) or a digital signalprocessor (DSP), encodes a detection signal (tactile signal) from eachtactile sensor 5 according to a predetermined data format, and collectsthe encoded tactile data Dc obtained by encoding in, for example, astorage device internally provided.

The reproduction device 4 is configured to include a computer devicesuch as a CPU and a DSP, and transmits the acquired encoded tactile dataDc to the decoding device 3. For example, the encoded tactile data Dccollected in the collection environment is acquired by the reproductiondevice 4 via a required network such as the Internet. Alternatively, theencoded tactile data Dc may be recorded in a portable recording medium,and the reproduction device 4 may acquire the encoded tactile data Dcvia the recording medium.

The decoding device 3 decodes the encoded tactile data Dc received fromthe reproduction device 4, and drives each tactile presentation device 6based on the tactile signal obtained by decoding.

The tactile presentation device 6 is a device that generates a tactilestimulus, and in the present example, a vibration device such as avibrator or an actuator is used.

In the present example, the respective tactile presentation devices 6are mounted on different parts of the human body of the recipient, andare each adapted to reproduce the tactile stimulus sensed by thecorresponding tactile sensor 5.

Here, in the present example, the tactile presentation devices 6 are inwired connection to the decoding device 3, and a configurationsurrounded by a broken line in the figure, that is, the configurationincluding the decoding device 3 and the tactile presentation devices 6,is to be mounted.

In the tactile reproduction system 1, the reproduction device 4 may beconfigured to have the function of the decoding device 3 so that thereproduction device 4 and the tactile presentation devices 6 are inwired connection to each other. However, with such a configuration,there is a risk of causing annoyance to the recipient on which thetactile presentation devices 6 are mounted. This annoyance is expectedto increase as the number of parts where tactile stimuli are to be givenincreases.

With the configuration of the tactile reproduction system 1 illustratedin FIG. 1 , it is possible to prevent such annoyance from being given tothe recipient.

The tactile reproduction system 1 illustrated in FIG. 1 is configured tobe a system that reproduces on the recipient the tactile sensation ofeach part perceived by a person on which the tactile sensors 5 aremounted, and is also configured to be a system applicable to the casewhere they are remote from each other.

2. Configuration of Encoding Device

FIG. 2 is a diagram illustrating of an internal configuration example ofthe encoding device 2. FIG. 2 also illustrates the tactile sensors 5illustrated in FIG. 1 together with the internal configuration exampleof the encoding device 2.

As illustrated, the encoding device 2 includes a plurality of amplifiers21, a plurality of analog/digital (A/D) converters 22, a preprocessingunit 23, an encoding unit 24, a control unit 25, a storage unit 26, acommunication unit 27, and a bus 28. The preprocessing unit 23, theencoding unit 24, the control unit 25, the storage unit 26, and thecommunication unit 27 are connected via the bus 28 so that they cancommunicate data with each other.

The detection signal from each tactile sensor 5 is input to thecorresponding amplifier 21 and adjusted to an appropriate dynamic range,and then input to the corresponding A/D converter 22 to be subjected toA/D conversion (analog/digital conversion).

Each detection signal subjected to the A/D conversion (that is, atactile signal for each part) is input to the preprocessing unit 23. Thepreprocessing unit 23 performs various types of digital signalprocessing such as noise reduction and calibration of the sensorcharacteristics of the tactile sensor 5. Each tactile signal subjectedto the signal processing by the preprocessing unit 23 is input to theencoding unit 24.

The encoding unit 24 includes, for example, a DSP, and encodes eachinput tactile signal according to a predetermined data format to obtainthe above-mentioned encoded tactile data Dc.

Details of encoding the tactile signal as the present embodiment will bedescribed again.

The control unit 25 is configured to include a microcomputer including,for example, a CPU, a read only memory (ROM), a random access memory(RAM), and the like, and performs the overall control of the encodingdevice 2 by executing, for example, processing according to a programstored in the ROM.

For example, the control unit 25 performs data communication with anexternal device via the communication unit 27.

The communication unit 27 is configured to perform data communicationwith an external device via a network such as the Internet, and thecontrol unit 25 is configured to perform data communication with theexternal device connected to the network via the communication unit 27.In particular, the control unit 25 is configured to cause thecommunication unit 27 to transmit the encoded tactile data Dc obtainedby the encoding unit 24 to the external device.

The storage unit 26 comprehensively represents a storage device such asa hard disk drive (HDD) or a solid state drive (SSD), and is used forvarious types of data storage in the encoding device 2. For example, thestorage unit 26 stores data necessary for control performed by thecontrol unit 25. Further, the encoded tactile data Dc obtained by theencoding unit 24 may be stored in the storage unit 26 under the controlof the control unit 25.

3. Configuration of Reproduction Device

FIG. 3 is a diagram illustrating an internal configuration example ofthe reproduction device 4. As illustrated, the reproduction device 4includes a control unit 41, a communication unit 42, a media drive 43, astorage unit 44, and a wireless communication unit 45, and also includesa bus 46 that connects these units so that they can communicate datawith each other.

The control unit 41 is configured to include, for example, amicrocomputer having a CPU, a ROM, a RAM, and the like, and controls theentire reproduction device 4.

The communication unit 42 is configured to perform data communicationwith an external device via a network such as the Internet.

The control unit 41 is configured to perform data communication with theexternal device connected to the network via the communication unit 42.In particular, the control unit 41 is configured to cause thecommunication unit 42 to receive the encoded tactile data Dc from anexternal device such as a server device on the network.

The media drive 43 is configured to allow a portable recording medium tobe loaded and unloaded, and is also configured as a reader/writer unitin which data can be written to and read from the loaded recordingmedium. Examples of the recording medium supported by the media drive 43include a memory card (for example, a portable flash memory), an opticaldisk recording medium, and the like. This media drive 43 makes itpossible to read out the encoded tactile data Dc recorded in theportable recording medium.

The storage unit 44 comprehensively represents a storage device such asan HDD or an SSD, and is used for various types of data storage in thereproduction device 4. For example, the storage unit 44 stores datanecessary for control performed by the control unit 41. Further, underthe control of the control unit 41, the encoded tactile data Dc read bythe media drive 43 and the encoded tactile data Dc received from theexternal device by the communication unit 42 can be stored in thestorage unit 44.

The wireless communication unit 45 performs short-range wirelesscommunication by a predetermined type of communication such as Bluetooth(registered trademark).

Here, the control unit 41 performs a control for causing thecommunication unit 42 to receive the encoded tactile data Dc and acontrol for causing the media drive 43 to read out the encoded tactiledata Dc, as parts of the above-mentioned overall control. The controlunit 41 also performs a control for causing the wireless communicationunit 45 to transmit the encoded tactile data Dc obtained via thecommunication unit 42 or the media drive 43 to the decoding device 3.

4. Configuration of Decoding Device

FIG. 4 is a diagram for describing an internal configuration example ofthe decoding device 3, and illustrates the tactile presentation devices6 together with the internal configuration example of the decodingdevice 3.

As illustrated, the decoding device 3 includes a plurality of amplifiers31, a plurality of digital/analog (D/A) converters 32, a combining unit33, a decoding unit 34, a control unit 35, a wireless communication unit36, a storage unit 37, and a bus 38. The combining unit 33, the decodingunit 34, the control unit 35, the wireless communication unit 36, andthe storage unit 37 are connected via the bus 38 so that they cancommunicate data with each other.

The control unit 35 is configured to include, for example, amicrocomputer having a CPU, a ROM, a RAM, and the like, and controls theentire decoding device 3.

The wireless communication unit 36 performs short-range wirelesscommunication by a method for communicating with the wirelesscommunication unit 45 in the reproduction device 4, such as Bluetooth.The encoded tactile data Dc transmitted from the reproduction device 4is received by the wireless communication unit 36.

The storage unit 37 is, for example, a storage device similar to thestorage unit 26 of the encoding device 2 or the storage unit 44 of thereproduction device 4, and is used for storing various types of dataused by the control unit 35 and the like.

The decoding unit 34 decodes the encoded tactile data Dc received viathe wireless communication unit 36 by a method described later to obtaina tactile signal for each part. The tactile signal for each partobtained by the decoding unit 34 is input to the combining unit 33.

The combining unit 33 performs signal processing, such as calibration ofthe tactile presentation device 6 and predetermined filter processing,on the received tactile signal for each part, if necessary.

Details of decoding the encoded tactile data Dc as the presentembodiment will be described again.

Each tactile signal that has passed through the combining unit 33 isinput to the corresponding D/A converter 32 to be subjected to D/Aconversion (digital/analog conversion), then adjusted to an appropriatedynamic range in the corresponding amplifier 31, and output to thecorresponding tactile presentation device 6.

As a result, each tactile presentation device 6 is driven based on thecorresponding tactile signal, and a tactile stimulus to be sensed in thedetection environment can be given to the recipient (that is, thetactile information can be reproduced).

Although only the tactile signal is mentioned above, a configuration canbe provided in which an audio signal and a video signal are collectedtogether with the tactile signal to provide the recipient with the soundand the image together with the tactile information.

5. Usage Examples of Tactile Reproduction System

Consider reproducing a content for presenting a tactile sensation inaddition to video. FIG. 5 is an illustrative diagram of a usage exampleof the tactile reproduction system 1.

In FIG. 5 , at the time of content production, in addition to the video,the encoded tactile data Dc in which the encoding device 2 encodes thetactile signals (including a body tactile signal 62, a finger tactilesignal 63, a foot tactile signal 64) collected from the tactile sensors5 (including a tactile sensor 5 b for the body, a tactile sensor 5 h forthe fingers, and the tactile sensor 5 f for the foot in the figure)mounted on a tactile collector 61 is recorded in synchronization withthe video, and the resulting data is stored as a content 65 in thestorage unit 44 of the reproduction device 4.

At the time of reproduction, the content 65 is transmitted from thestorage unit 44 to the decoding device 3 through the wirelesscommunication unit 45, and the content 65 received by the wirelesscommunication unit 36 is decoded by the decoding unit 34. As a result,it is possible to present the tactile signals (including a body tactilesignal 66, a finger tactile signal 67, a foot tactile signal 68)corresponding to the tactile presentation devices 6 (including a tactilepresentation device 6 b for the body, a tactile presentation device 6 hfor the fingers, and a tactile presentation device 6 f for the foot inthe figure) mounted on a recipient 69, through the tactile presentationdevices 6.

Examples of scenes where the tactile presentation is actually performedin the video include a scene in which a character beats (or is beaten),a scene where a character shoots (or is shot) with a gun, a scene wherea character receives a blast, and a scene where a character feels theshaking of the ground.

In the figure, as an example of the tactile signals in the actualcontent 65, a waveform example 50 of the body, finger, and foot tactilesignals is illustrated. In the waveform example 50, the tactile signalsobtained by the tactile sensors 5 (the tactile sensor 5 b, the tactilesensor 5 h, the tactile sensor 5 f) mounted on the three parts, thebody, fingers, and foot of the tactile collector 61 are illustrated.

Describing in chronological order, first, vibration caused by thereaction due to bullet firing is generated on the fingers in the scenewhere the tactile collector 61 shoots an opponent with a gun, thenvibration caused by the impact of the shot body is generated in the bodyin the scene where the opponent also shoots at the tactile collector'sbody with a gun, and after that, vibration of the ground is graduallypropagated to the foot, body, and fingers in the scene where anearthquake occurs. For example, when such a content is reproduced, therecipient 69 can feel high-quality reality by reproducing the tactilesensation with the vibration in addition to the video and audio.

6. Tactile Reproduction Method as Embodiment

[6-1. Issues Related to Tactile Signal Transmission]

Hereinafter, a tactile reproduction method as an embodiment will bedescribed. First, the tactile reproduction method as the embodiment is amethod focusing on human tactile characteristics.

A vibration detection threshold curve illustrated in FIG. 6 has beenreported as a guideline for human tactile sensitivity. In the figure,the horizontal axis represents the frequency and the vertical axisrepresents the magnitude of a tactile stimulus (vibration: displacementin this example). The vibration detection threshold curve in FIG. 6 isbased on the experimental results of “Four channels mediate themechanical aspects of touch S. J., Bolanowski 1988”.

The illustrated vibration detection threshold curve is an example ofexperimentally examining whether or not human beings feel the vibrationas a tactile sensation, that is, a tactile sensitivity. Human beingscannot perceive vibrations lower than this curve as tactile sensations.

As can be seen from FIG. 6 , the frequency with the highest tactilesensitivity of human beings is generally about 200 Hz. Therefore,devices and applications that generate vibrations are often designed togenerate vibrations up to about 200 Hz.

On the other hand, although not shown in the results of FIG. 6 , it iscommonly known that human beings can perceive vibrations with afrequency of up to about 1 kHz as a tactile sensation. Human beings canperceive vibrations with a frequency component of about 1 kHz andvibrations without a frequency component of about 1 kHz as differenttactile sensations.

For example, the vibration generated when a bottle is uncorked includesvibrations with frequencies of up to several kHz. When a user receives avibration transmitted from a device that presents a tactile sensation asbeing up to several hundred Hz, the user cannot perceive that vibrationas a tactile sensation when a bottle is uncorked with a sufficient senseof reality. Therefore, in order to give the user a tactile sensationwith a deeper sense of reality, it is necessary to perform a tactilepresentation with a vibration with frequencies of up to about 1 kHz.

However, as the width of frequencies included in a signal increases, theamount of data of the signal increases, so that a delay in transmissionand reception of the signal is highly likely to occur. As a result, ifthe quality of a tactile sensation is improved, a situation may occur inwhich the tactile sensation cannot be presented at an appropriatetiming.

The transmission and reception of a signal and a delay in tactilepresentation will be described taking specific examples.

First, the amount of tactile signal data will be described. When atactile signal is transmitted between devices, it is first converted todigital data. The capacity of digital data is represented by the numberof bits required per unit time, that is, a bit rate B. Incidentally, thetactile sensitivity depends not only on the frequency of vibration butalso on the amplitude. For example, the above-mentioned experimentalresults of FIG. 6 show that vibrations perceivable by human beings havean amplitude of about 50 dB (−20 dB to 30 dB) or more and a frequency ofabout 1000 Hz. In the following, it is assumed that the amplitude ofvibration is about 70 dB, taking into consideration the actualdistribution of tactile information perceivable by human beings.

When a tactile signal is digitized using Linear Pulse Code Modulation(LPCM), the amplitude of vibration that can be expressed by 1 bit is 6dB. Accordingly, 12 bits are required for an amplitude of vibration of70 dB. On the other hand, when the frequency of vibration is 1000 Hz,the sampling frequency needs to be doubled to 2000 Hz, and the bit rateB0 is obtained by the following [Equation 1].

B0=12 bit/sample×2000 sample/sec=24 kbit/sec  [Equation 1]

This value is very small compared with, for example, the bit rate ofCompact Disc (CD)=700 kbps/ch, which is a typical format for audiosignals. Accordingly, even if such a tactile signal is additionallyincorporated into some system, it seems to be unlikely to cause a bigproblem.

However, as described above, it is known that the band of a tactilesignal perceivable by human beings extends to several kHz. For example,when a tactile signal is reproduced with up to 2000 Hz, the bit rate is48 kbit/sec, which is a double of that of [Equation 1].

The tactile sensation is present everywhere on the surface of the humanbody, unlike the sense of sight (two eyes) and the sense of hearing (twoears). Considering only the fingertips of both hands, there are 10places, and when all their tactile signals are to be handled, the bitrate will be further increased by 10 times, which is 480 kbit/sec. Asthe places of interest increase to knuckles, palms, and so on, the bitrate increases dramatically.

Although a tactile signal is basically a one-dimensional signal, thephysical phenomenon of vibration can be defined on three axes (x, y, z).If all of such axes are to be handled, a bit rate of 1440 kbit/sec,which is three times as much. This value is as high as more than 1411kbit/sec for audio CD.

As described above, the total amount of tactile signal data requiredincreases as the reproducibility of a tactile stimulus is improved andthe number of user's parts given the tactile stimulus increases.Accordingly, the increase in the total amount of data becomes a heavyload on the network system that transmits the tactile signal.

Now consider a vibration generated by the motion of “tracing an objectwith the hand”, which is significant in tactile sensation. This motioncan be divided into three phases: a person touching the object, thenmoving the hand along the surface of the object, and finally releasingthe hand from the object. Focusing on the vibration waveforms generatedby this series of motions, the “moment of touch” and the “moment ofrelease” are very short in time, but each have a sharp change inintensity. On the other hand, the vibration waveform during the motionof the hand along the surface of the object has a relatively long timescale and is steady.

In the present specification, a state of a part to be given a tactilesignal touching an object, including a person touching the object andthen moving the hand along the surface of the object until the personfinally releases the hand from the object, is defined as a touch state.In addition, the state where a part to be given a tactile signal is nottouching the object is defined as a non-touch state.

The section of a tactile signal generated by a series of motions of“tracing an object with the hand”, that is, a signal section indicatinga touch state with the object in the tactile signal is defined as atouch signal section.

In addition, in the touch signal section, a signal section including theboundary between the touch state and the non-touch state with theobject, such as “the moment when the hand touches the object” or “themoment when the hand leaves the object”, is defined as a first signalsection. Further, in the touch signal section, a signal section exceptfor the first signal section, such as “the state where the hand isplaced on the object” or “the state where the hand is moving along thesurface of the object”, is defined as a second signal section.

FIG. 7 is a diagram illustrating a vibration waveform 70 of a tactilesignal when a person traces an object 79 with a hand 78.

In the vibration waveform 70, a sharp pulse waveform like a signalsection 73 is observed at the moment when the hand 78 illustrated in ascene 72 touches the object 79 after the state where the hand 78illustrated in a scene 71 leaves the object 79. It is known that thevibration intensity, timing, pitch, waveform attenuation rate, and thelike at this “moment of touch” may affect the perception of the hardnessof the object and how fast the person touches the object 79.

The signal section 73 including the boundary between the touch state andthe non-touch state of the hand 78 and the object 79 is defined as thefirst signal section. This signal section 73 is very short in time.

On the other hand, as illustrated in the subsequent scene 75, for thehand 78 being moved on the object 79, the waveform of a signal section74 with a gradual change in amplitude is observed. The vibration duringthis period affects the texture perception of the object 79 such asslippery and rough, and the pitch and amplitude of the waveform aresignificant factors.

The signal section 74 in the state where the touch state of the hand 78and the object 79 is maintained is defined as the second signal section.

Finally, as illustrated in a scene 76, when the hand 78 leaves theobject 79, the waveform of the corresponding signal section 77 is shortin time and has a sharp change in intensity, and it is presumed to haveproperties similar to the waveform at the time of touch in the signalsection 73. The signal section 77 including the boundary between thetouch state and the non-touch state of the hand 78 and the object 79 isdefined as the first signal section.

The series of signal sections including the signal section 73, thesignal section 74, and the signal section 77 are referred to as a touchsignal section 200 indicating a touch state with the object 79 in thetactile signal.

Now dividing the signal into the signal section 73 and the signalsection 77 (first signal sections), each having a sharp change inintensity and the steady signal section 74 (second signal section), thefirst signal section has more features to be preserved in terms of thenature of perception and the features are also more complicated.Accordingly, it is desirable to adopt a high-quality encoding methodusing a relatively high bit-rate, for example, waveform encoding such asAdvanced Audio Codec (AAC) and lossless compression encoding such asFree Lossless Audio Codec (FLAC).

On the other hand, the latter steady second signal section has a gradualchange over time. Accordingly, in the second signal section, there islittle information loss even if long-time signals are collectivelyencoded, there are few features to be preserved, and the features arealso simple. Therefore, it is expected that human beings are less likelyto be aware of signal deterioration even if a method that emphasizessignal energy reproduction with high compression rate, such as CodeExcited Linear Prediction (CELP) or Harmonic Vector Excitation Coding(HVXC); or a low-quality encoding method using a lower bit rate than thehigh-quality encoding method, such as parametric encoding thatsynthesizes equivalent signals from the above-mentioned features to bepreserved, is used.

Therefore, in order to reproduce well the short time section such as thesignal section 73 and the signal section 77, if the long time signalsection 74 is also encoded by the high-quality encoding method, thetactile signal has an amount of redundant information as a whole,resulting in a reduced compression efficiency. An increase in the totalamount of data due to such a reduced compression efficiency also causesa delay in transmission and reception of the tactile signal.

In addition, as a factor of delay, a factor other than the total amountof data as described above can be considered.

For example, when a tactile signal is transmitted through wirelesscommunication, the encoded tactile data Dc for the tactile signal may belost due to interference on the transmission path or the like. When dataloss occurs, the data is retransmitted from the device on thetransmitting side, and there may be a delay in the time required for thereceiving side to complete the reception of the data. In other words, asthe capacity of the data to be retransmitted increases, the timerequired for retransmitting the data corresponding to the data lossincreases, and as a result, the time when the transmission and receptionof the tactile signal is successfully completed may be further delayed.

As described above, if the time when the transmission of a tactilesignal is completed is delayed, the reproducibility of the tactilesensation may be deteriorated. Specifically, a tactile stimulus is notgiven to the user (recipient) at an appropriate timing, so that asituation may occur in which the tactile stimulus fails to besynchronized with a content related to other sensations such as videoand sound.

Next, the application of a specific type of wireless communication inconsideration of the above-mentioned situation will be described.

Since the device for performing the tactile presentation is installed soas to be in touch with the user, it is generally desired to communicatewirelessly with other devices from the viewpoint of the weight of thedevice. However, when a wideband type of wireless connection such asWi-Fi (registered trademark) is used, the battery of the device becomeslarge from the viewpoint of power consumption, so that the convenienceof the user may be reduced. Further, when Wi-Fi is used, it generallytakes a processing time for the procedures from a signal transmissionrequest of the transmitting side device to the receiving processing ofthe receiving side device, so that there may be a larger delay ascompared with other types of wireless connection.

On the other hand, a short-range type of wireless connection such asBluetooth makes it possible to perform communication with low powerconsumption and low delay as compared with other types of wirelessconnection, so that it may be suitable for transmission of a tactilesignal. However, in the short-range type of wireless connection, theallowable amount of data that can be transmitted at one time is lessthan that of other types of wireless connection. For example, intransmitting a content that gives a tactile stimulus to a user whilesynchronizing with video and sound, a situation may occur in which thecommunication capacity allocated to the transmission of a tactile signalis not sufficient.

In addition, in a service for streaming video and audio via theInternet, when an additional tactile sensation is to be transmitted tothe user, for example, a situation may occur in which the communicationcapacity allocated to the transmission of a tactile signal is notsufficient due to the Quality of Service (QoS) function according to theline condition of the network.

In view of the above circumstances, the present embodiment aims toachieve both low delay of the tactile signal and suppression ofdeterioration of the tactile reproducibility by reducing the amount ofdata to be transmitted without impairing the tactile reproducibility asmuch as possible.

[6-2. Encoding Method]

FIG. 8 is a functional block diagram illustrating the functions of theencoding unit 24. As illustrated, the encoding unit 24 includes a signalinput unit 80, a first buffering processing unit 86, a determinationunit 82, a first encoding unit 83, a second buffering processing unit87, a second encoding unit 84, and an encoded signal output unit 85.

A tactile signal 81 to be encoded is input to the signal input unit 80from the preprocessing unit 23.

The first buffering processing unit 86 includes a first buffer memoryused for determining a first signal section and a second signal sectionto buffer the tactile signal 81 input to the signal input unit 80.

As illustrated in FIG. 7 , the determination unit 82 determines a firstsignal section (signal section 73, 77) which is a part of the touchsignal section 200 in the tactile signal and includes the boundarybetween the touch state and the non-touch state of the object 79 and thehand 78, and a second signal section (signal section 74) which is asignal section except for the first signal section in the touch signalsection 200, for example.

The determination unit 82 determines the first signal section based on,for example, the result of performing waveform analysis on the tactilesignal 81. The determination unit 82 also determines that the signalsection other than the first signal section is the second signalsection.

A typical waveform analysis in determining the first signal section issignal rise detection based on an amplitude change rate of the tactilesignal. The rising edge of this signal is carried out by a known signalprocessing method.

Here, the determination unit 82 can determine the first signal sectionby performing attack detection known in the fields of signal intensitychange rate, frequency spectrum, and other signal processing.

The determination unit 82 can also determine the first signal sectionbased on the distance between the object and the tactile sensor or therecognition of the occurrence of touch, which is detected by variousother types of sensors. Further, the determination unit 82 may determinean encoding method using metadata (flagged encoding quality) given bythe creator (operator).

Furthermore, as the determination unit 82, a determination device may beused that performs machine learning with a large number of vibrationsignals to automatically determine the target section from the inputwaveform.

The determination of the first signal section by the determination unit82 can extend the advantageous effects of the present technology togeneral signals corresponding to the moment of touch of a person withthe object 79 and the moment of release of the person from the object 79in tactile sensation.

When it is determined that the whole or a part of the tactile signal 81is the first signal section, the determination unit 82 transmits, to thefirst encoding unit 83, the corresponding section if it is a part or thewhole tactile signal 81 if it is the whole.

The determination unit 82 also transmits the second signal section ofthe tactile signal 81 that is not transmitted to the first encoding unit83 to the second buffering processing unit 87.

The first encoding unit 83 encodes the tactile signal in the firstsignal section received from the determination unit 82 by a high bitrate encoding method to generate first encoded data (encoded tactiledata Dc). In the following description, the high bit rate encodingmethod is also referred to as a high-quality encoding method.

Here, the high-quality encoding method used for the first signal sectionis desirably a method using a relatively high rate that can restorecomplex waveform information, such as signal pulse height, width, andtiming, and energy change rate, with high time resolution. For example,waveform encoding such as AAC and lossless compression encoding such asFLAC are applicable.

The second buffering processing unit 87 includes a second buffer memoryto buffer the signal received according to the result of determining thesignal section from the determination unit 82.

The second encoding unit 84 encodes the signal in the second signalsection buffered by the second buffering processing unit 87 by anencoding method using a bit rate lower than that of the high-qualityencoding method for the first signal section, to generate second encodeddata (encoded tactile data Dc). In the following description, theencoding method using the bit rate lower than that of the high-qualityencoding method is also referred to as a low-quality encoding method.

Here, in the low-quality encoding method used for the second signalsection, it is desirable to encode a small number of parameters, such asenergy and pitch, with a long time frame. Specifically, such encoding isa method that emphasizes signal energy and pitch reproduction with highcompression rate, such as CELP and HVXC, or a parametric encoding methodthat synthesizes an equivalent signal from the above-mentioned featuresto be preserved.

Of such parameters, for example, an energy can be calculated by thefollowing [Equation 2] using an effective value of a signal f[n].

[Math.1] $\begin{matrix}{E = \sqrt{\frac{\sum_{n = 0}^{N - 1}{f\lbrack n\rbrack}^{2}}{N}}} & \left\lbrack {{Equation}2} \right\rbrack\end{matrix}$

A typical method for acquiring a pitch is a method of obtaining a peakpoint m_0 of an autocorrelation function for signals as represented bythe following [Equation 3] and using the reciprocal of the correspondingtime as a pitch frequency.

[Math.2] $\begin{matrix}{{R\lbrack m\rbrack} = {\sum\limits_{n = 0}^{N - 1}{{f\lbrack n\rbrack}{f\left\lbrack {n - m} \right\rbrack}}}} & \left\lbrack {{Equation}3} \right\rbrack\end{matrix}$

The encoded signal output unit 85 outputs the first encoded datagenerated by the first encoding unit 83 and the second encoded datagenerated by the second encoding unit 84.

FIG. 9 illustrates a data format example of the encoded tactile data Dc.In FIG. 9 , a data format for one frame of the encoded tactile data Dcis illustrated as an example.

An identifier 100 is stored at the beginning of a frame header added tothe signal as associated information, the sampling frequency of thesignal is recorded in a field 101, and the quantization resolution ofthe signal is recorded in a field 102. Further, in a field 103, anencoding method ID (Identification) for identifying whether the data hasbeen encoded by a high-quality encoding method or a low-quality encodingmethod is stored. For example, “0” is stored for the high-qualityencoding method, and “1” is stored for the low-quality encoding method.

In a field 104, the number of signal samples included in this frame isstored, and in a field 105, signal data is stored.

With reference to the flowchart of FIG. 10 , an example of a processingprocedure for implementing the encoding method as the embodimentdescribed above will be described. The processing illustrated in FIG. 10is processing of generating the encoded tactile data Dc for one frame,and is repeatedly executed for each frame.

The processing for implementing the encoding method in the encoding unit24 may be implemented as software processing. Alternatively, it may beimplemented by hardware or a combination of software and hardware.

First, the encoding unit 24 acquires a tactile signal to be encoded fromthe preprocessing unit 23 in step S110.

In step S111 following step S110, the encoding unit 24 buffers theacquired tactile signal in order to determine a first signal section anda second signal section. This is the processing corresponding to theabove-mentioned first buffering processing unit 86.

In step S112 following step S111, the encoding unit 24 determineswhether the signal section of the acquired tactile signal is a firstsignal section to be encoded using the high-quality encoding method or asecond signal section to be encoded using the low-quality encodingmethod.

At this time, the encoding unit 24 determines, for example, based on acomparison between the amplitude change rate of the tactile signal and apredetermined threshold value, a rising edge of the signal asillustrated in the signal section 73 of FIG. 7 and a falling edge of thesignal as illustrated in the signal section 77. The encoding unit 24determines the signal section in which such a rising edge or fallingedge of the signal is detected, as the first signal section. Theencoding unit 24 also determines the signal section in which a risingedge or a falling edge of the signal is not detected, as the secondsignal section.

The encoding unit 24 may also determine a rising edge or a falling edgeof the signal based on an intensity distribution in which the tactilesignal is decomposed into its frequency components by using thefrequency spectrum of the tactile signal.

The encoding unit 24 may also determine the first signal section and thesecond signal section based on section information on the first signalsection and the second signal section added to the tactile signal 81.The section information is given as metadata by the work of a creator,for example, as described above.

An example of using metadata added to the tactile signal to determinethe first signal section and the second signal section will now bedescribed.

FIG. 11 illustrates a data format example of the encoded tactile dataDc. FIG. 11 illustrates a data format for one frame of a tactile signalinput to the encoding unit 24. Here, a WAVE file is used as an exampleof the data format.

In the WAVE file, header information including format information suchas sampling frequency, quantization resolution, and the number ofchannels is stored in a field 130 at the beginning, and signal data isstored in a field 131.

In the present embodiment, metadata for determining the first signalsection to be encoded by the high-quality encoding method or the secondsignal section to be encoded by the low-quality encoding method is addedto the end of this data. In the following description, the encoding bythe high-quality encoding method is also referred to as high-qualityencoding, and the encoding by the low-quality encoding method is alsoreferred to as low-quality encoding.

Here, for a channel for a target to be subjected to the high-qualityencoding, an example is illustrated in which the number of samples fromthe beginning of the file is assigned at each of the start point and theend point of the first signal section. A channel ID of the target to besubjected to the high-quality encoding is stored in a field 132, thenumber of start samples is stored in a field 133, and the number of endsamples to be subjected to the high-quality encoding is stored in afield 134. When there are a plurality of first signal sections to besubjected to the high-quality encoding, a set of fields 132, 133, and134 may be added as many as the number of sections.

In that case, the encoding unit 24 checks in advance the end data(fields 132, 133, 134) of the signal input to be encoded in step S112 ofFIG. 10 , and determines whether or not the currently buffered signalsection is the signal section specified by the metadata.

If the encoding unit 24 determines that it is the first signal sectionto be subjected to the high-quality encoding, the processing proceedsfrom step S112 to step S113 to perform the high-quality encoding on thefirst signal section. In other words, the encoding unit 24 performsencoding by an encoding method using a bit rate higher than in thelow-quality encoding.

In step S114 following step S113, the encoding unit 24 generates firstencoding data to which an encoding method ID corresponding to thehigh-quality encoding method is assigned.

Thereafter, the processing in the encoding unit 24 proceeds to stepS118.

On the other hand, if the encoding unit 24 determines in step S112 thatit is the second signal section other than the first signal section tobe subjected to the high-quality encoding, the processing proceeds tostep S115, and then the tactile signal is buffered until it enables thelow-quality encoding.

In the present embodiment, the high-quality encoding uses a shorterconversion length, and the low-quality encoding uses a longer conversionlength than in the high-quality encoding. Therefore, when it isdetermined that the high-quality encoding is unnecessary, the tactilesignal may not be buffered enough to enable the low-quality encoding.

Accordingly, in step S115, the encoding unit 24 buffers the tactilesignal for the number of samples required by the low-quality encoding.This is the processing corresponding to the above-mentioned secondbuffering processing unit 87.

In step S116 following step S115, the encoding unit 24 performs thelow-quality encoding on the second signal section. Specifically, theencoding unit 24 performs encoding by an encoding method using a lowerbit rate than in the high-quality encoding.

In step S117 following step S116, the encoding unit 24 generates secondencoding data to which an encoding method ID corresponding to theencoding method for the low-quality encoding is assigned.

Thereafter, the processing in the encoding unit 24 proceeds to stepS118.

In step S118, the encoding unit 24 determines whether or not the entiresignal has been encoded, that is, whether or not the processing ofgenerating the first encoded data and the second encoded data for allchannels for the tactile signal has been completed.

When the processing for all signal sections is not completed, theprocessing in the encoding unit 24 returns to step S111. As a result,the same processing as for one frame is executed for the next section.

On the other hand, when the processing for all signal sections iscompleted, a series of processing illustrated in FIG. 11 ends in theencoding unit 24. When there are a plurality of channels, the encodingunit 24 executes the processing illustrated in FIG. 11 for all thechannels.

In the present embodiment, an example in which the conversion length ofthe low-quality encoding method is longer than that of the high-qualityencoding method has been described. However, the present technology canalso be applied to the case where the conversion lengths of thelow-quality encoding method and the high-quality encoding method areequal.

In this case, the encoding unit 24 always buffers a number of samplesequal to the conversion length in step S111, and applies only one typeof encoding quality to each signal buffered at one time. As a result,the buffering performed in step S115 prior to encoding by thelow-quality encoding in step S116 is not necessary as illustrated inFIG. 12 . By making the conversion lengths of the low-quality encodingand the high-quality encoding equal in this way, the processing load ofthe encoding unit 24 is reduced.

[6-3. Decoding Method]

The functions of the combining unit 33 and the decoding unit 34 in thedecoding device 3 will be described with reference to FIG. 4 .

The decoding unit 34 includes a first decoding unit 34 a and a seconddecoding unit 34 b.

The first decoding unit 34 a decodes the first encoded data obtained byperforming the high-quality encoding on the first signal section whichis a part of the touch signal section. The first decoding unit 34 atransmits first waveform data obtained by decoding the first encodeddata to the combining unit 33.

The second decoding unit 34 b decodes the second encoded data obtainedby performing the low-quality encoding on the second signal sectionwhich is a signal section except for the first signal section. Thesecond decoding unit 34 b transmits second waveform data obtained bydecoding the second encoded data to the combining unit 33.

The combining unit 33 combines the first waveform data obtained bydecoding the first encoded data and the second waveform data obtained bydecoding the second encoded data.

The combining unit 33 determines whether the waveform data currentlybeing output is that has been encoded by an encoding method differentfrom that for the currently decoded waveform data waiting to be output.

If their encoding methods are the same, the combining unit 33appropriately post-processes and outputs the signal according to theencoding method. Specifically, in this post-processing, the waveformdata waiting to be output and the waveform data being output aresubjected to combining processing to be normally performed in theencoding method, while combining processing including cross-fadedescribed later is not performed.

On the other hand, if their encoding methods are different, the numberof signal samples and the handling of termination may be differentbetween the encoding methods, and if the signal is output as it is, anunpleasant discontinuous waveform may occur. Therefore, when decodeddata following the waveform data encoded by a different encoding methodis output, the combining unit 33 performs signal processing so that thetwo data are smoothly combined.

The combining unit 33 performs cross-fade processing on a combinedportion between the first waveform data and the second waveform datawhich have originally been encoded by different encoding methods, forexample.

As illustrated in FIG. 13 , when first waveform data 91 which hasoriginally been subjected to the high-quality encoding and secondwaveform data 92 which has originally been subjected to the low-qualityencoding are continuously output, a section 93 where the first waveformdata 91 and the second waveform data 92 overlap is defined, and datapieces in that section are each multiplied and added up with acoefficient such that the total of the data pieces becomes “1”, so thatit is possible to smoothly combine the first waveform data 91 and thesecond waveform data 92 to obtain an output waveform 94. The sameapplies to the case where the second waveform data 92 which hasoriginally been subjected to the low-quality encoding and the firstwaveform data 91 which has originally been subjected to the high-qualityencoding are continuously output.

The combining unit 33 may use known processing other than the cross-fadeprocessing for the processing of combining the first encoded data andthe second waveform data.

With reference to the flowchart of FIG. 14 , an example of a processingprocedure for implementing the decoding method as the embodimentdescribed above will be described. The processing illustrated in FIG. 14is processing of decoding the encoded tactile data Dc for one frame toobtain a tactile signal, and is repeatedly executed for each frame.

The processing for implementing the encoding method in the decodingdevice 3 may be implemented as software processing. Alternatively, itmay be implemented by hardware or a combination of software andhardware.

First, the decoding unit 34 of the decoding device 3 acquires theencoded tactile data Dc in step S140. In step S141 following step S140,the decoding unit 34 acquires the encoding method ID from the encodedtactile data Dc.

In step S142 following step S141, the decoding unit 34 determines basedon the acquired encoding method ID whether the encoded tactile data Dcis the first encoded data encoded by the high-quality encoding method orthe second encoded data encoded by the low-quality encoding method.

If the encoding method ID indicates encoding by the high-qualityencoding method, the processing in the decoding unit 34 proceeds fromstep S142 to step S143 to decode the first encoded data by a decodingmethod corresponding to the high-quality encoding method. The decodingunit 34 transmits first waveform data obtained by decoding the firstencoded data to the combining unit 33.

Thereafter, the processing in the decoding unit 34 proceeds from stepS143 to step S145.

On the other hand, if the encoding method ID indicates encoding by thelow-quality encoding method, the processing in the decoding unit 34proceeds from step S142 to step S144 to decode the second encoded databy a decoding method corresponding to the low-quality encoding method.The decoding unit 34 transmits second waveform data obtained by decodingthe second encoded data to the combining unit 33.

Thereafter, the processing in the decoding unit 34 proceeds from stepS144 to step S145.

In step S145, the combining unit 33 of the decoding device 3 determineswhether or not the encoding method for the waveform data currentlywaiting to be output decoded by the decoding unit 34 and the encodingmethod for the waveform data currently being output are different fromeach other.

If the encoding methods are different, the processing in the combiningunit 33 proceeds from step S145 to step S146 to perform the cross-fadeprocessing as illustrated in FIG. 9 on the combined portion between thetwo pieces of waveform data.

Details of the cross-fade processing executed by the combining unit 33will now be described with reference to FIGS. 15 and 16 .

As illustrated in FIG. 15 , the combining unit 33 passes a decodedsignal 150 decoded in step S143 or step S144 of FIG. 14 through a block151 that delays the decoded signal 150 by one frame by buffering or thelike.

Then, the combining unit 33 compares the delayed signal (waveform datawaiting to be output) passing through the block 151 with the latestdecoded signal 150 (waveform data currently being output).

If the encoding methods for the delayed signal and the decoded signal150 are the same, the combining unit 33 combines the latest decodedsignal 150 with the delayed signal by normal combining processing asillustrated in FIG. 15 to generate a tactile output 147. An additionunit 155 in the figure represents that the decoded signal 150 iscombined to the delayed signal in such a way.

If the encoding methods for the delayed signal and the decoded signal150 are different, the combining unit 33 performs cross-fade processingillustrated in FIG. 16 . The delayed signal and the decoded signal 150each have a margin, which is a portion where their frames overlap. Thismargin portion referred to as a combining margin portion.

The cross-fade processing uses a cross-fade function 152 multiplied bythe decoded signal 150 and a cross-fade function 153 multiplied by thedelayed signal. For the fade-in and fade-out portions of the cross-fadefunctions 152 and 153, any window function having a total of “1” can beused such as a triangular window and a Hanning window. The cross-fadefunction 152 is a function that transitions from “0” to “1” in thecombining margin portion with the passage of time and maintains “1”after the combining margin portion. The cross-fade function 153 is afunction that transitions from “1” to “0” in the combining marginportion with the passage of time.

The combining unit 33 multiplies the decoded signal 150 by thecross-fade function 152 by a multiplication unit 156, and multiplies thedelayed signal by the cross-fade function 153 by a multiplication unit157. The combining unit 33 generates a tactile output 147 by adding themultiplication results by an addition unit 158.

After the cross-fade processing in step S146 of FIG. 14 , the processingin the combining unit 33 proceeds to step S147.

When the encoding methods are not different, that is, the encodingmethod for the currently received waveform data and the encoding methodfor the next received waveform data are the same, the processing in thecombining unit 33 proceeds to step S147 without performing thecross-fade processing in step S146. This is because, in general, anyencoding method is originally designed so that signals are smoothlycombined even at the joint of the units to be converted, and it is notnecessary to perform additional processing for combining the signals.

The combining unit 33 outputs the decoded tactile signal in step S147.In step S148 following step S147, the decoding unit 34 determineswhether or not all the encoded tactile data Dc have been decoded, thatis, whether or not the decoding processing has been completed for theencoded tactile data Dc for all signal sections of the tactile signal.

When the processing for all signal sections is not completed, theprocessing in the decoding unit 34 returns to step S140. As a result,the same processing as for one frame is executed for the remainingsection(s).

On the other hand, when the processing for all signal sections iscompleted, a series of processing illustrated in FIG. 14 ends in thedecoding unit 34. When there are a plurality of channels, the decodingunit 34 executes the processing illustrated in FIG. 14 for all thechannels.

7. First Application Example

As a first application example of the present technology, consider atactile movie in which vibration as a tactile sensation is presented toa viewer, in addition to the video and audio as illustrated in FIG. 17 .

First, at the stage of creation, vibration is collected in addition tovideo and audio. In this collection, for example, the vibrationperceived by a character in the movie is recorded, for example, by usinga tactile sensor 161 (tactile sensor 5) mounted on the body of an actor160. In another example, vibrations generated in the environment arerecorded by an installation on the floor of the collecting site.

The vibration waveform collected (recorded) in this way is encoded by anencoding device 162 (encoding device 2). The details of the encoding areas described with reference to FIGS. 8, 10 and 12 , and after thecontrol of the encoding quality involving the analysis of the signal andthe use of meta information, the section(s) with a sharp change insignal strength is/are encoded by the high-quality encoding method andthe other section(s) is/are encoded by the low-quality encoding method.After the video and audio are also encoded by a known method, theencoded video/audio/vibration data is stored in a storage 163 typifiedby a DVD (Digital Versatile Disc) or the like.

For example, in the case where a character traces a wall with the hand,it is desirable for the moment of touch to adopt the high-qualityencoding because the vibration waveform has a large change over time; itis sufficient for the period during which the character moves the handalong the wall to adopt the low-quality encoding.

Therefore, when a tactile signal is input by the tactile sensor 161mounted on the hand of the actor 160, a known signal processing methodis applied to the tactile signal to detect the rising edge of thesignal, and the high-quality encoding is performed only on that section(first signal section). Further, at the moment when a creator wishes togive a sensation of touch with a wall while viewing captured images, thecreator may add such meta information to the tactile signal. In thiscase, the determination unit 82 of FIG. 8 detects this meta information,and performs the high-quality encoding only on the tactile signal atthat time. As a result, the encoded tactile data Dc can be obtained inwhich the high-quality encoding is performed only at the moment of touchwith the wall and the low-quality encoding is performed in other times.

As another example, vibrations generated by a touch with a wall may notbe collected during shooting of a movie, and instead, a creator maydesign a waveform for giving a sensation as if a touch with the walloccurs. Even for this case, the encoding method can be determined by asignal processing method and meta information additionally given by thecreator as described above.

A viewer 164 watches a movie from the storage 163 by a player 165.Meanwhile, the player 165 decodes and outputs the video and audio from aknown video and audio output device by a known method.

The corresponding vibration signal(s) is/are also decoded by a decodingdevice 166 (decoding device 3) by the method described with reference toFIG. 14 , and then output from a vibration output device(s) 167. Inother words, the signals are decoded for the respective sectionsaccording to the corresponding encoding method ID assigned at the timeof encoding, the signal switching processing is performed so as toobtain a smooth signal as necessary, and the resulting signal is outputas a decoded signal.

At this time, a plurality of vibration signal systems may be outputcorresponding to the number of systems stored in the storage, or anumber of vibration signal systems may be output adjusted according tothe number of devices owned by the viewer 164. A known channel numberconversion method can be used for this adjustment.

In the present embodiment, it is possible to save the storage requiredfor storing the vibration data. As a result, the advantageous effects ofa reduced storage cost, a further increased number of channels ofvibration, an increased resolution, and an increased storage used forvideo and audio can be obtained.

8. Second Application Example

As a second application example of the present technology, consider acloud game in which all operations of a user 174 as illustrated in FIG.18 are transmitted from an edge 176 to a cloud 172, all the changes invideo, audio, and tactile sensation in response to the operations arecalculated and processed on the cloud 172, and the results aretransmitted to the edge 176 (returning to the user 174) and reproducedthere.

A general home video game, that is, an example in which a console isplaced in front of the user 174, button operations of the user 174 aretransmitted to the console, and various signals processed there aretransmitted to various output devices through wired connection orthrough very short-range wireless connection, has the same problem,configuration, and effects as the tactile movie described in the firstapplication example, and thus, the description thereof will be omitted.

Since there is no high-performance computer on the edge 176 in cloudgames, the interpretation of complicated user operations and thesynthesis and adjustment of feedback based on the interpretation are allperformed on the cloud 172, and the resulting large capacity data suchas video, audio, and tactile sensation is transmitted from the cloud 172to the edge 176.

At this time, if the communication band from the cloud 172 to the edge176 is not sufficiently secured, noise and signal interruption occur onthe edge 176 side, which greatly impairs the user experience. Inaddition, the vibration in tactile sensation is different depending onhow the same object is touched, and accordingly, a method of storingsuch different vibrations as separate signals requires a huge amount ofstorage. Although a method of storing only the reference signal andmodulating that signal according to how the object is touched may beused, the former method is expected to obtain a higher quality of outputsignal.

The second application example will be described. It is now assumed thatvideo and audio are handled by a known method, and only the tactilesensation will be described in this application example. First, at thestage of creating a game, a recorded or synthesized vibration signal 170(tactile signal) is prepared. Examples of such vibration include thevibration of shooting with a gun, the vibration of opening a door, thevibration of tracing a wall, the vibration of being hit on the shoulder,and so on. It may also be a vibration that is difficult to perceive,such as heartbeat. Each of these signals is encoded by the encodingdevice 171 (encoding device 2).

In this encoding, as described above, quality control and thecorresponding encoding method for each section are applied. The encodedtactile data Dc thus obtained is stored in a storage 173 on the cloud172.

Next, when the user 174 plays this game, an operation of the user 174 isfirst input from a controller 175 and transmitted from the edge 176 tothe cloud 172. On the cloud 172, the operation of the user 174 isreceived by a transmission and reception unit 177, and interpreted by acontrol unit 178 to select and read out the encoded tactile data Dc ofthe vibration signal to be output from the storage 173.

The read encoded tactile data Dc is transmitted to the edge 176 as itis, decoded by a decoding device 179 (decoding device 3) at the edge176, and output from a vibration output device 1710.

As a result, the amount of information of the encoded tactile data Dc tobe recorded is reduced, so that it is possible to reduce the storagecapacity on the cloud 172. In addition, communicating the data asencoded and then decoded on the edge 176 side makes it possible toreduce the amount of communication data, so that the user 174 can enjoyexperience of a realistic tactile sensation with vibration signals in acloud game, even with a narrow communication band between the cloud 172and the edge 176.

This application example is simple because the decoding processingincludes only reading an encoding method ID and the correspondingdecoding, and there is also an advantage that a high-performancecomputer is not required on the edge 176 side.

As a modification example of the second application example, theencoding quality may be adjusted according to the communication band.Before the encoded tactile data Dc read from the storage 173 istransmitted as it is from the transmission and reception unit 177, thecontrol unit 178 acquires the communication band information between thecloud 172 and the edge 176.

If the current communication environment is excellent, the encodedtactile data Dc is transmitted as it is from the transmission andreception unit 177. On the other hand, if the communication environmentis poor, the read encoded tactile data Dc is encoded again by theencoding device 171. At this time, the control unit 178 provides newencoded tactile data Dc suitable for the communication environment.

For example, if it is determined that the communication environment ispoor when the encoded tactile data Dc of a vibration signalcorresponding to tracing a wall is transmitted, for example, a method inwhich a longer time frame is used to encode the second signal section onwhich the low-quality encoding is performed while the hand is movedalong the wall and thus to reduce the frequency of parameter updatesmakes it possible to further reduce the amount of data at the expense ofsome quality.

In the second application example of the present technology, theseparation between the first signal section to be subjected to thehigh-quality encoding and the second signal section to be subjected tothe low-quality encoding is based on human perceptual characteristics,and accordingly, such adjustment is easily performed while maintaininghuman perception. For example, since the above adjustment is for thesection where the time resolution is originally considered not to be akey factor, the influence due to even the adjustment of the frequency ofparameter updates is restrictive.

9. Third Application Example

As a third application example of the present technology, consider anexample in which, in mail-order sales via the Internet as illustrated inFIG. 19 , the texture of a product 180 is reproduced as a vibration tomake a user 186 feel tactile information on the product 180. Forexample, when the user 186 finds a product on a shopping site, avibration device 1810 at hand can output a vibration as its detailedinformation, so that the user 186 can confirm the vibration as thetexture of the product 180.

First, in order to record the texture of the product 180, a vibrationsensor 181 (tactile sensor 5) is used to record the vibrations whenvarious touches are made. At this time, for example, a robot arm 182 maybe used for repeated touch motions.

Each piece of vibration signal data acquired by the vibration sensor 181is encoded by an encoding device 183 depending on the correspondingtouch motion. This encoding method is as described above, and in thisapplication example, a touch sensor 184 (tactile sensor 5) isadditionally attached to the robot arm 182, and the quality of encodingis controlled based on the information from the touch sensor 184.

For example, the moment when the robot arm 182 touches the product 180and the moment when the robot arm 182 leaves the product 180 can beidentified by the touch sensor 184.

It is predicted that the change in the signal over time is sharp for acertain period from that moment. Accordingly, that section is set as thefirst signal section to be subjected to the high-precision encoding. Inthis case, since the quality control is completed by the informationfrom the touch sensor 184, signal processing is unnecessary. Then, theresulting pieces of encoded tactile data Dc are stored in a storage 185.

Next, a transmission and reception unit 187 receives a reproductionrequest from the user 186, and a control unit 188 reads out theappropriate encoded tactile data Dc from the storage 185 and transmitsit to the user 186 side. A decoding device 189 (decoding device 3) onthe receiving user 186 side decodes the encoded tactile data Dc, andreproduces the vibration from a vibration device 1810 held by the user186.

As a result, the user 186 can feel the texture of the product 180 overthe Internet. In addition, due to the advantageous effects of thisapplication example, the data capacity is reduced, so that the vibrationdata can be downloaded in a short time, and the load on the web page isalso reduced, so that the quality of the user experience is notimpaired.

10. Summary

The decoding device 3 according to the above-described embodimentincludes: the first decoding unit 34 a that decodes first encoded dataobtained by encoding (high-quality encoding) a first signal section (forexample, the signal section 73 or 77) with a first bit rate, the firstsignal section being a part of the touch signal section 200 which is asignal section indicating a touch state with an object in a tactilesignal, the first signal section being a signal section including aboundary between the touch state and a non-touch state with the object;and the second decoding unit 34 b that decodes second encoded dataobtained by encoding (low-quality encoding) a second signal section (forexample, the signal section 74) with a bit rate lower than the first bitrate, the second signal section being a signal section except for thefirst signal section in the touch signal section 200 (see S140 to S144in FIGS. 4, 7, and 14 ).

The encoded tactile data Dc in which only the significant section in thetouch signal section is encoded (high-quality encoded) with the high bitrate and the other section is encoded (low-quality encoded) with the lowbit rate is decoded, so that both a reduced amount of data to betransmitted and a guarantee of reproduction of tactile sensation can beachieved.

Thus, it is possible to receive the data, in which the amount of theentire encoded tactile data Dc to be transmitted is reduced, with lowdelay without impairing the tactile reproducibility as much as possible.Therefore, it is possible for the recipient to experience a high-qualitytactile sensation without feeling a delay.

The decoding device 3 according to the embodiment includes the combiningunit 33 that combines first waveform data obtained by decoding the firstencoded data and second waveform data obtained by decoding the secondencoded data (see S145 and S146 in FIGS. 4 and 14 ).

As a result, the pieces of waveform data are integrated into one pieceof waveform data.

Therefore, it is possible to prevent the generation of an unpleasantdiscontinuous waveform caused by outputting pieces of waveform dataseparately, so that the tactile reproducibility can be improved.

In the decoding device 3 according to the embodiment, the combining unit33 performs cross-fade processing on a combined portion between thefirst waveform data and the second waveform data (see S146 in FIG. 14 ,and FIGS. 15 and 16 ). As a result, the first waveform data and thesecond waveform data are smoothly combined.

Therefore, it is possible to prevent the generation of an unpleasantdiscontinuous waveform caused by outputting pieces of waveform dataseparately, so that the tactile reproducibility can be improved.

The decoding method according to the embodiment includes: decoding firstencoded data obtained by encoding (high-quality encoding) a first signalsection with a first bit rate, the first signal section being a part ofa touch signal section (for example, the signal section 73 or 77) whichis a signal section indicating a touch state with an object in a tactilesignal, the first signal section being a signal section including aboundary between the touch state and a non-touch state with the object;and decoding second encoded data obtained by encoding (low-qualityencoding) a second signal section (for example, the signal section 74)with a bit rate lower than the first bit rate, the second signal sectionbeing a signal section except for the first signal section in the touchsignal section.

Also according to such a decoding method as an embodiment, the sameoperation and effect as those of the decoding device 3 as theabove-described embodiment can also be obtained.

The encoding device 2 according to the embodiment includes: thedetermination unit 82 that determines a first signal section (forexample, the signal section 73 or 77) and a second signal section (forexample, the signal section 74), the first signal section being a partof a touch signal section 200 which is a signal section indicating atouch state with an object in a tactile signal, the first signal sectionbeing a signal section including a boundary between the touch state anda non-touch state with the object, the second signal section being asignal section except for the first signal section in the touch signalsection 200; the first encoding unit 83 that encodes (high-qualityencodes) the first signal section with a first bit rate; and the secondencoding unit 84 that encodes (low-quality encodes) the second signalsection with a bit rate lower than the first bit rate (see FIGS. 7, 8,and 10 ). Only the significant section in the touch signal section isencoded with the high bit rate, and the other section is encoded withthe low bit rate, so that both a reduced amount of data to betransmitted and a guarantee of reproduction of tactile sensation can beachieved.

Thus, it is possible to reduce the amount of the entire encoded tactiledata Dc to be transmitted without impairing the tactile reproducibilityas much as possible. Therefore, it is possible to transmit the encodedtactile data Dc with low delay while maintaining the perceptual qualityof the tactile signal.

In the encoding device 2 according to the embodiment, the determinationunit 82 determines the first signal section and the second signalsection based on section information on the first signal section and thesecond signal section added to the tactile signal (see S112 in FIG. 10 ,and FIG. 11 ).

As a result, processing such as waveform analysis is not necessary indetermining the first signal section and the second signal section.

Therefore, the processing load of the encoding device 2 for determiningthe first signal section and the second signal section can be reduced,so that it is possible to perform more efficient encoding processing.

In the encoding device 2 according to the embodiment, the determinationunit 82 determines the first signal section and the second signalsection based on a result of performing a waveform analysis on thetactile signal (see S112 in FIG. 10 ). As a result, information fordifferentiating between the first signal section and the second signalsection is not required to be added to the touch signal in individuallyencoding the first signal section and the second signal section.

Therefore, it is possible to reduce the burden on the operator whocreates the touch signal to which section information is added for eachsignal section.

In the encoding device 2 according to the embodiment, the determinationunit 82 determines the first signal section and the second signalsection based on an amplitude change rate of the tactile signal (seeS112 in FIG. 10 ).

As a result, it is possible to appropriately determine the first signalsection with a sharp change in the waveform of the tactile signal.

Therefore, it is possible to appropriately achieve both a reduced amountof data and a guarantee of reproduction of tactile sensation.

In the encoding device 2 according to the embodiment, the secondencoding unit 84 encodes the second signal section by an encoding methodusing a longer conversion length than in encoding for the first signalsection (see S116 in FIG. 10 ).

As a result, the amount of second encoded data in the second signalsection is reduced.

Therefore, it is possible to reduce the amount of the entire encodedtactile data Dc to be transmitted.

In the encoding device 2 according to the embodiment, the secondencoding unit 84 performs parametric encoding on the second signalsection (see S116 in FIG. 10 ).

As a result, the amount of second encoded data in the second signalsection is reduced.

Therefore, it is possible to reduce the amount of the entire encodedtactile data Dc to be transmitted.

The encoding device 2 according to the embodiment includes a firstbuffer memory used for determining the first signal section and thesecond signal section, and a second buffer memory used for encoding thesecond signal section according to a result of determining (see S111 andS115 in FIGS. 8 and 10 ).

As a result, it is possible for the second buffering processing unit 87to buffer a tactile signal that fails to be buffered by the firstbuffering processing unit 86 for the tactile signal in the second signalsection for which the conversion length is longer than in the encodingfor the first signal section.

Therefore, it is not necessary to match the buffering period with thesecond signal section having a long conversion length, so that it ispossible to prevent the processing time of the high-quality encoding forthe tactile signal of the first signal section from being delayed andthus to improve the efficiency of the encoding processing.

The encoding method according to the embodiment includes: determining afirst signal section (for example, the signal section 73 or 77) and asecond signal section (for example, the signal section 74), the firstsignal section being a part of a touch signal section 200 which is asignal section indicating a touch state with an object in a tactilesignal, the first signal section being a signal section including aboundary between the touch state and a non-touch state with the object,the second signal section being a signal section except for the firstsignal section in the touch signal section 200; encoding (high-qualityencoding) the first signal section with a first bit rate; and encoding(low-quality encoding) the second signal section with a bit rate lowerthan the first bit rate.

Also according to such an encoding method as an embodiment, the sameoperation and effect as those of the encoding device 2 as theabove-described embodiment can also be obtained.

Here, the functions of the decoding device 3 and the encoding device 2described so far can be implemented as software processing by, forexample, a CPU, a DSP, or the like. The software processing is executedbased on a program.

A first program as an embodiment is a program causing an informationprocessing device to execute: a function of decoding first encoded dataobtained by encoding a first signal section with a first bit rate, thefirst signal section being a part of a touch signal section which is asignal section indicating a touch state with an object in a tactilesignal, the first signal section being a signal section including aboundary between the touch state and a non-touch state with the object;and a function of decoding second encoded data obtained by encoding asecond signal section with a bit rate lower than the first bit rate, thesecond signal section being a signal section except for the first signalsection in the touch signal section.

Such a first program can realize the decoding device 3 according to theabove-described embodiment.

A second program as an embodiment is a program causing an informationprocessing device to execute: a function of determining a first signalsection and a second signal section, the first signal section being apart of a touch signal section which is a signal section indicating atouch state with an object in a tactile signal, the first signal sectionbeing a signal section including a boundary between the touch state anda non-touch state with the object, the second signal section being asignal section except for the first signal section in the touch signalsection; a function of encoding the first signal section with a firstbit rate; and a function of encoding the second signal section with abit rate lower than the first bit rate. Such a second program canrealize the encoding device 2 according to the above-describedembodiment.

The first and second programs as described above can be recorded inadvance in a recording medium embedded in a device such as a computerdevice or a ROM or the like in a microcomputer that includes a CPU.

Alternatively, the programs can be stored (recorded) temporarily orperpetually on a removable recording medium such as a flexible disc, acompact disc read-only memory (CD-ROM), a magnet optical (MO) disc, aDVD, a Blu-ray Disc (registered trademark), a magnetic disk, asemiconductor memory, or a memory card. The removable recording mediumcan be provided as so-called package software.

The first and second programs can be installed from the removablerecording medium to a personal computer and can also be downloaded froma download site via a network such as the Internet or a local areanetwork (LAN).

Further, the first and second programs are suitable for a wide range ofprovision of the decoding device and the encoding device according tothe embodiments. For example, the programs are downloaded to a personalcomputer, a portable information processing device, a mobile phone, agame console, an audio and visual (AV) device, or the like, making itpossible for the personal computer or the like to function as thedecoding device and the encoding device of the present technology.

11. Present Technology

The present technology can be configured as follows.

(1)

A decoding device, including:

a first decoding unit that decodes first encoded data obtained byencoding a first signal section with a first bit rate, the first signalsection being a part of a touch signal section which is a signal sectionindicating a touch state with an object in a tactile signal, the firstsignal section being a signal section including a boundary between thetouch state and a non-touch state with the object; and

a second decoding unit that decodes second encoded data obtained byencoding a second signal section with a bit rate lower than the firstbit rate, the second signal section being a signal section except forthe first signal section in the touch signal section.

(2)

The decoding device according to (1), including a combining unit thatcombines first waveform data obtained by decoding the first encoded dataand second waveform data obtained by decoding the second encoded data.

(3)

The decoding device according to (2), wherein the combining unitperforms cross-fade processing on a combined portion between the firstwaveform data and the second waveform data.

(4)

A decoding method, including:

decoding first encoded data obtained by encoding a first signal sectionwith a first bit rate, the first signal section being a part of a touchsignal section which is a signal section indicating a touch state withan object in a tactile signal, the first signal section being a signalsection including a boundary between the touch state and a non-touchstate with the object; and

decoding second encoded data obtained by encoding a second signalsection with a bit rate lower than the first bit rate, the second signalsection being a signal section except for the first signal section inthe touch signal section.

(5)

A program causing an information processing device to execute;

a function of decoding first encoded data obtained by encoding a firstsignal section with a first bit rate, the first signal section being apart of a touch signal section which is a signal section indicating atouch state with an object in a tactile signal, the first signal sectionbeing a signal section including a boundary between the touch state anda non-touch state with the object; and

a function of decoding second encoded data obtained by encoding a secondsignal section with a bit rate lower than the first bit rate, the secondsignal section being a signal section except for the first signalsection in the touch signal section.

(6)

An encoding device, including:

a determination unit that determines a first signal section and a secondsignal section, the first signal section being a part of a touch signalsection which is a signal section indicating a touch state with anobject in a tactile signal, the first signal section being a signalsection including a boundary between the touch state and a non-touchstate with the object, the second signal section being a signal sectionexcept for the first signal section in the touch signal section;

a first encoding unit that encodes the first signal section with a firstbit rate; and

a second encoding unit that encodes the second signal section with a bitrate lower than the first bit rate.

(7)

The encoding device according to (6), wherein the determination unitdetermines the first signal section and the second signal section basedon section information on the first signal section and the second signalsection added to the tactile signal.

(8)

The encoding device according to (6) or (7), wherein the determinationunit determines the first signal section and the second signal sectionbased on a result of performing a waveform analysis on the tactilesignal.

(9)

The encoding device according to any one of (6) to (8), wherein thedetermination unit determines the first signal section and the secondsignal section based on an amplitude change rate of the tactile signal.

(10)

The encoding device according to any one of (6) to (9), wherein thesecond encoding unit encodes the second signal section by an encodingmethod using a longer conversion length than in encoding for the firstsignal section.

(11)

The encoding device according to any one of (6) to (10), wherein thesecond encoding unit performs parametric encoding on the second signalsection.

(12)

The encoding device according to (10) including:

a first buffer memory used for determining the first signal section andthe second signal section, and

a second buffer memory used for encoding the second signal sectionaccording to a result of determining.

(13)

An encoding method, including:

determining a first signal section and a second signal section, thefirst signal section being a part of a touch signal section which is asignal section indicating a touch state with an object in a tactilesignal, the first signal section being a signal section including aboundary between the touch state and a non-touch state with the object,the second signal section being a signal section except for the firstsignal section in the touch signal section; encoding the first signalsection with a first bit rate; and encoding the second signal sectionwith a bit rate lower than the first bit rate.

(14)

A program causing an information processing device to execute:

a function of determining a first signal section and a second signalsection, the first signal section being a part of a touch signal sectionwhich is a signal section indicating a touch state with an object in atactile signal, the first signal section being a signal sectionincluding a boundary between the touch state and a non-touch state withthe object, the second signal section being a signal section except forthe first signal section in the touch signal section;

a function of encoding the first signal section with a first bit rate;and a function of encoding the second signal section with a bit ratelower than the first bit rate.

Finally, the advantageous effects described in the present disclosureare exemplary and not limited, and may have other advantageous effectsor may have part of the advantageous effects described in the presentdisclosure.

The embodiments described in the present disclosure are merely examples,and the present technology is not limited to the above-describedembodiments. Therefore, it goes without saying that various changesaside from the above-described embodiments can be made according to thedesign and the like within a scope that does not depart from thetechnical spirit of the present technology. It should be noted that notall combinations of configurations described in the embodiments areessential for solving the problem.

REFERENCE SIGNS LIST

-   1 Tactile reproduction system-   2 Encoding device-   3 Decoding device-   24 Encoding unit-   33 Combining unit-   34 Decoding unit-   34 a First decoding unit-   34 b Second decoding unit-   82 Determination unit-   83 First encoding unit-   84 Second encoding unit-   86 First buffering unit-   87 Second buffering unit-   200 Touch signal section

1. A decoding device, comprising: a first decoding unit that decodesfirst encoded data obtained by encoding a first signal section with afirst bit rate, the first signal section being a part of a touch signalsection which is a signal section indicating a touch state with anobject in a tactile signal, the first signal section being a signalsection including a boundary between the touch state and a non-touchstate with the object; and a second decoding unit that decodes secondencoded data obtained by encoding a second signal section with a bitrate lower than the first bit rate, the second signal section being asignal section except for the first signal section in the touch signalsection.
 2. The decoding device according to claim 1, comprising acombining unit that combines first waveform data obtained by decodingthe first encoded data and second waveform data obtained by decoding thesecond encoded data.
 3. The decoding device according to claim 2,wherein the combining unit performs cross-fade processing on a combinedportion between the first waveform data and the second waveform data. 4.A decoding method, comprising: decoding first encoded data obtained byencoding a first signal section with a first bit rate, the first signalsection being a part of a touch signal section which is a signal sectionindicating a touch state with an object in a tactile signal, the firstsignal section being a signal section including a boundary between thetouch state and a non-touch state with the object; and decoding secondencoded data obtained by encoding a second signal section with a bitrate lower than the first bit rate, the second signal section being asignal section except for the first signal section in the touch signalsection.
 5. A program causing an information processing device toexecute; a function of decoding first encoded data obtained by encodinga first signal section with a first bit rate, the first signal sectionbeing a part of a touch signal section which is a signal sectionindicating a touch state with an object in a tactile signal, the firstsignal section being a signal section including a boundary between thetouch state and a non-touch state with the object; and a function ofdecoding second encoded data obtained by encoding a second signalsection with a bit rate lower than the first bit rate, the second signalsection being a signal section except for the first signal section inthe touch signal section.
 6. An encoding device, comprising: adetermination unit that determines a first signal section and a secondsignal section, the first signal section being a part of a touch signalsection which is a signal section indicating a touch state with anobject in a tactile signal, the first signal section being a signalsection including a boundary between the touch state and a non-touchstate with the object, the second signal section being a signal sectionexcept for the first signal section in the touch signal section; a firstencoding unit that encodes the first signal section with a first bitrate; and a second encoding unit that encodes the second signal sectionwith a bit rate lower than the first bit rate.
 7. The encoding deviceaccording to claim 6, wherein the determination unit determines thefirst signal section and the second signal section based on sectioninformation on the first signal section and the second signal sectionadded to the tactile signal.
 8. The encoding device according to claim6, wherein the determination unit determines the first signal sectionand the second signal section based on a result of performing a waveformanalysis on the tactile signal.
 9. The encoding device according toclaim 6, wherein the determination unit determines the first signalsection and the second signal section based on an amplitude change rateof the tactile signal.
 10. The encoding device according to claim 6,wherein the second encoding unit encodes the second signal section by anencoding method using a longer conversion length than in encoding forthe first signal section.
 11. The encoding device according to claim 6,wherein the second encoding unit performs parametric encoding on thesecond signal section.
 12. The encoding device according to claim 10,comprising: a first buffer memory used for determining the first signalsection and the second signal section, and a second buffer memory usedfor encoding the second signal section according to a result ofdetermining.
 13. An encoding method, comprising: determining a firstsignal section and a second signal section, the first signal sectionbeing a part of a touch signal section which is a signal sectionindicating a touch state with an object in a tactile signal, the firstsignal section being a signal section including a boundary between thetouch state and a non-touch state with the object, the second signalsection being a signal section except for the first signal section inthe touch signal section; encoding the first signal section with a firstbit rate; and encoding the second signal section with a bit rate lowerthan the first bit rate.
 14. A program causing an information processingdevice to execute: a function of determining a first signal section anda second signal section, the first signal section being a part of atouch signal section which is a signal section indicating a touch statewith an object in a tactile signal, the first signal section being asignal section including a boundary between the touch state and anon-touch state with the object, the second signal section being asignal section except for the first signal section in the touch signalsection; a function of encoding the first signal section with a firstbit rate; and a function of encoding the second signal section with abit rate lower than the first bit rate.